001562US SPEC 



MEDICAL IMAGING AND NAVIGATION SYSTEM 
Gera STROMMER and Uzi EICHLER 

5 CROSS REFERENCE INFORMATION 

This application is a Continuation-in-Part of application no. 
09/314,474, filed May 18, 1999. 

FIELD OF THE INVENTION 

10 The present invention relates to medical diagnostic and surgery 

systems and methods in general, and to methods and systems for 
three-dimensional medical imaging and navigation, in particular. 

BACKGROUND OF THE INVENTION 

is Methods and systems for acquiring and presenting 

two-dimensional and three-dimensional images are known in an art. 
Three-dimensional imaging enhances modem diagnostics, therapy and 
surgery procedures. 

A two-dimensional imaging system processes and represents 

20 two-dimensional internal body slices in static or in dynamic form on a 
display. A conventional two-dimensional ultrasound imaging system 
includes an ultrasound transducer, an image capturing module and an 
image-processing unit. 

The ultrasound transducer is placed in close proximity to the 

25 tissue to be examined. The ultrasound transducer converts an electrical 
signal to ultrasonic waves and directs the waves toward the examined 
tissue. The ultrasonic waves are in part absorbed, dispersed, refracted 
and reflected. The ultrasound transducer detects the ultrasonic reflections. 
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The ultrasound transducer converts the reflected ultrasonic waves to an 
electrical signal and provides it to the image-processing unit. 

The image-processing unit processes the received electrical 
signal, thereby producing a plurality of two-dimensional images of slices of 
5 the inspected tissue. The image-capturing module captures each 
two-dimensional image and can provide each of them to a display or a 
printer. 

US Patent No. 5,152,290 to Freeland, entitled "Method for 
recording ultrasound images to diagnose heart and coronary artery 

10 disease" is directed to a method for capturing and displaying 
two-dimensional ultrasound images of the heart for diagnosing heart 
disease, such as coronary artery disease. The method disclosed by 
Freeland includes the steps of detecting an electrocardiogram (ECG) 
signal after peak exercise, detecting the two-dimensional images of the 

15 heart, storing selected images, each with the ECG reading at the time that 
the image was taken and displaying a quad-image group. The system 
detects and records a two-dimensional image sequence continuously at a 
rate of at least eight images per heartbeat. 

US Patent No. 5,690,113, issued to Sliwa, Jr. et al., entitled 

20 "Method and apparatus for two-dimensional ultrasonic imaging" is directed 
to a method and apparatus for generating a two-dimensional ultrasonic 
image using a hand-held single element transducer probe, having a fixed 
scan-line. The system provides displaying two-dimensional ultrasonic 
images of the body of a patient. This system detects two-dimensional 

25 ultrasonic images, and determines the spatial location and orientation of 
the ultrasound transducer, at the same time. The system includes a probe 
with an ultrasound transducer, capable of imaging a single scan-line and a 
means for tracking the spatial location and orientation of the ultrasound 
transducer. The scan-line is fixed in an orientation and spatial position 

30 relative to the movable transducer. The system further includes a 

computing means, which computes the spatial location and the orientation 
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of each scan-line as the transducer is moved. Thereby, the scan-lines are 
presented as a complete image. Alternatively, an electromagnetic 
transmitter and receiving sensor determine the spatial orientation and 
position of each scan-line in free space. 
5 A typical three-dimensional ultrasound imaging system includes 

a conventional two-dimensional ultrasound imaging system, a location and 
orientation detection system, an image processing system and a 
displaying system. Such systems provide three-dimensional imaging of 
internal organs such as the liver, kidneys, gallbladder, breast, eyes, brain, 
10 and the like. 

The location and orientation detection system provides the 
location and orientation of ultrasound transducer. The location and 
orientation of each of the captured two-dimensional images are 
determined from the location and orientation of the transducer. 
; ! 15 The image processing system reconstructs a three-dimensional 

image of the inspected organ, by processing the captured two-dimensional 
images, each according to the location and orientation thereof. Finally, the 
displaying system displays the received three-dimensional image of the 
inspected organ. 

\ 20 US Patent No. 5,787,889 issued to Edwards et al., and entitled 

"Ultrasound imaging with real time 3D image reconstruction and 
visualization" is directed to generation and visualization of 
three-dimensional ultrasound images. The method disclosed by Edwards 
includes the following steps: acquiring data, reconstructing a volume, and 

25 visualizing an image. The system provides for achieving and visualizing 
three-dimensional ultrasound images with a two-dimensional ultrasound 
medical imaging system included therein. An operator can perform various 
visualization tasks on the reconstructed three-dimensional image, such as 
rotating the image in different viewing angles and plans. 

so Another type of three-dimensional imaging system, which is 

known in the art, is operative to produce a motion picture of the heart or 
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the lungs. This system Includes a conventional two-dimensional ultrasound 
imaging system, an ECG monitor, a location and orientation detection 
system, an image processor and a display system. The ECG monitor 
detects the timing signal of the heart. The ECG timing signal is used to 

5 synchronize or trigger the recording of the two-dimensional images 
representative of selected points in the ECG timing signal. The ultrasound 
transducer detects two-dimensional ultrasound images of the heart at any 
given moment (e.g., at a selected point of time on ECG timing signal). 
Each two-dimensional image represents a specific slice of the heart 

10 according to the specific activity-state thereof. The location and orientation 
of each of the two-dimensional images are directly determined from the 
location and orientation of the transducer. 

The image processor reconstructs a three-dimensional image of 
the heart from captured two-dimensional images having the same 

15 activity-state. Finally, the display system displays a sequence of the 
reconstructed images, thereby presenting a three-dimensional motion 
picture of the heart. 

US Patent No. 5,924,989 issued to Polz T and entitled "Method 
and device for capturing diagnostically acceptable three-dimensional 

20 ultrasound image data records", is directed to a method and a system for 
generating a three-dimensional image sequence of the heart. This system 
includes a three-dimensional ultrasound imaging system, combined with 
an echocardiograph. The system detects two-dimensional ultrasound 
images and stores each of them together with the location and orientation 

25 thereof and with the organ cycle location as provided by the 
echocardiogram, at the time that the image was acquired. 

Utilizing a special algorithm, the system reconstructs a 
three-dimensional image from all of the two-dimensional images having 
the same organ cycle location, and displays a sequence of the 

30 reconstructed three-dimensional images. 
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SUMMARY OF THE PRESENT INVENTION 

It is an object of the present invention to provide a novel method 
and system for medical in-vivo invasive probing. In accordance with the 
present invention, there is thus provided a medical imaging and navigation 
system. The system includes a processor, a display unit, a database, a 
medical positioning system (MPS), a two-dimensional imaging system, an 
inspected organ monitor interface, and a superimposing processor. 

The MPS includes a transducer MPS sensor and a surgical tool 
MPS sensor. The two-dimensional imaging system includes an imaging 
transducer. The processor is connected to the display unit, the database, 
the MPS, the two-dimensional imaging system, the inspected organ 
monitor interface, and to the superimposing processor. The inspected 
organ monitor interface is further connected to an organ monitor. The 
surgical tool MPS sensor is firmly attached to a surgical tool. The 
transducer MPS sensor is firmly attached to the imaging transducer. The 
organ monitor monitors an organ timing signal associated with an 
inspected organ. The system reconstructs a plurality of three-dimensional 
images from a plurality of detected two-dimensional images, according to 
the respective location and orientation of each two-dimensional image and 
its position within the inspected organ timing signal. Since the all of the 
MPS sensors belong to the same MPS system, the system provides the 
location and orientation of the surgical tool, within the same coordinate 
system of the detected two-dimensional images. 

In accordance with another aspect of the present invention, 

there is thus provided a medical imaging and navigation system. The 

system includes a processor, a display unit, a database, an MPS, an 

inspected organ monitor, interface and a superimposing processor. The 

processor is connected to the display unit, the database, the MPS, the 

inspected organ monitor interface and to the superimposing processor. 

The inspected organ monitor interface is connected to an organ monitor. 

The MPS includes a surgical tool MPS sensor being firmly attached to a 
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surgical tool. The organ monitor monitors an organ timing signal 
associated with an inspected organ. This system is adapted to operate on 
p re-stored images. 

In accordance with a further aspect of the present invention, 
there is thus provided a method for displaying an image sequence of a 
moving inspected organ. The method includes the steps of detecting an 
organ timing signal of the inspected organ, detecting a plurality of 
two-dimensional images of the inspected organ using an image detector, 
and detecting the location and orientation of the image detector. The 
method further includes the steps of associating each of the 
two-dimensional images with the image detector location and orientation 
and with the detected organ timing signal, and reconstructing a plurality of 
three-dimensional images from the two-dimensional images. The method 
further includes the steps of selecting one of the three-dimensional images 
according to a real-time reading of the organ timing signal, and displaying 
the selected three-dimensional image. 

In accordance with another aspect of the present invention, 
there is thus provided a method for displaying an image sequence of a 
moving inspected organ. The method includes the steps of detecting an 
organ timing signal of the inspected organ, and selecting one of a 
previously stored three-dimensional images according to a real-time 
reading of the organ timing signal. The method further includes the steps 
of detecting the location and orientation of a surgical tool, superimposing a 
representation of the surgical tool onto the selected three-dimensional 
image, and displaying the superimposed three-dimensional image. 

in accordance with a further aspect of the present invention, 
there is thus provided a method for displaying an image sequence of a 
moving inspected organ. The method includes the steps of detecting an 
organ timing signal of the inspected organ, detecting the location and 
orientation of a point of view of a user and selecting one of a previously 

stored three-dimensional images according to a real-time reading of the 
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organ timing signal. The method further includes the steps of rendering the 
selected three-dimensional image according to the detected location and 
orientation of the point of view and displaying the selected 
three-dimensional image. 

5 In accordance with another aspect of the present invention, 

there is thus provided a method for displaying an image sequence of a 
moving inspected organ. Each image in the image sequence is associated 
with the location and orientation of the image within a predetermined 
coordinate system. The method includes the steps of detecting an organ 

10 timing signal of the inspected organ, selecting one of a previously stored 
two-dimensional images according to a real-time reading of the organ 
timing signal and displaying the selected two-dimensional image. This 
system is adapted for a two-dimensional imaging and displaying 
environment. 

15 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more 
fully from the following detailed description taken in conjunction with the 
drawings in which: 

5 Figure 1 is a schematic illustration of a multi functional 

three-dimensional imaging system, constructed and operative in 
accordance with a preferred embodiment of the present invention; 

Figure 2A is an illustration in perspective of an inner-body radial 
ultrasound imaging system, constructed and operative in accordance with 
10 another preferred embodiment of the present invention; 

Figure 2B is an illustration in perspective of a plurality of radial 
two-dimensional images of the walls of an inspected vessel; 

Figure 2C is a schematic illustration in detail of the MPS system 
of Figure 1, constructed and operative in accordance with a further 
15 preferred embodiment of the present invention; 

Figure 3 is a schematic illustration of a two-dimensional image in 
a given coordinate system; 

Figure 4 is an illustration in perspective of a plurality of 
two-dimensional images and an organ timing signal; 
20 Figure 5A is a schematic illustration of a plurality of 

three-dimensional volumes, according to another preferred embodiment of 
the present invention; 

Figure 5B is a schematic illustration of some of the 
three-dimensional volumes of Figure 5A, at a later stage of image 
25 reconstruction; 

Figure 5C is a schematic illustration of a selected 
three-dimensional volume of Figure 5A, going through a procedure of 
image updating; 

Figure 5D is a schematic illustration of a two-dimensional image 
30 which includes foreign object information; 
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Figure 6 is a schematic illustration of a method for operating the 
three-dimensional imaging system of Figure 1, operative in accordance 
with a further preferred embodiment of the presented invention; 

Figure 7A is a schematic illustration of a real-time 
5 three-dimensional display playback system, constructed and operative in 
accordance with another preferred embodiment of the present invention; 

Figure 7B is a schematic illustration of the goggles of Figure 7A, 
displaying a three-dimensional image of the heart of a patient; 

Figure 8 is an illustration in perspective of a cyclical sequence of 
10 three-dimensional images, according to a further preferred embodiment of 
the present invention; 

Figure 9 is a schematic illustration of a superimposing process, 
operative in accordance with another preferred embodiment of the present 
invention; 

15 Figure 10 is a schematic illustration of a method for operating 

the system of Figure 7A, operative in accordance with a further preferred 
embodiment of the present invention; 

Figure 11 is an illustration in perspective of an inner-vascular 
imaging and surgery system, constructed and operative in accordance 
20 with another preferred embodiment of the invention; and 

Figure 12 is a schematic illustration of a multi function 
two-dimensional imaging system, constructed and operative in accordance 
with a further preferred embodiment of the present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

The present invention overcomes the disadvantages of the prior 
art by providing methods and systems for constructing and displaying 
three-dimensional images of moving organs, synchronously with the 
movement of these organs and synchronously with an invasive tool, such 
as a catheter. According to a preferred embodiment, the three-dimensional 
images and the presentation of the invasive tool, all reside within a single 
coordinate system, and no registration of a plurality of coordinate systems 
is required. 

According to one aspect of the invention there is provided a 
pseudo real time imaging system, for minimal invasive surgery. This 
system includes a two-dimensional image acquisition system, a medical 
positioning system (MPS) which is basically a location and orientation 
detection system, a specific organ monitor and an image processing 
system. The location and orientation detection system includes at least 
three sensors. The first sensor is mounted on the image detector of the 
two-dimensional image acquisition system. The second sensor is mounted 
on the minimal invasive surgery tool. The third sensor is attached to the 
body of the patient for reference. Attaching the third sensor is the body of 
the patient during all of the steps of the method of the disclosed technique, 
assures that both the sensor of the image detector and the sensor of the 
surgical tool, as well as additional sensors which are attached to further 
modules of the system of the invention, remain in a single coordinate 
system at all times. 

The system acquires two-dimensional images at a rate, faster 
than the organ cycle frequency, and preferably equal to the organ cycle 
frequency multiplied by a natural number. The system records each 
two-dimensional image, acquired by the image detector, in association 
with the detected location and orientation thereof, and with the organ 
timing signal reading, as detected by the organ monitor. It is noted that the 
system operates under the assumption that the detected organ is 
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characterized by a cyclic behavior, and that a reading in one cycle is likely 
to be detected ih subsequent cycles. 

The imaging system reconstructs a three-dimensional image 
from all of the recorded two-dimensional images, which have the same 

5 organ timing signal reading (from different cycles). When the 
reconstructed three-dimensional images include sufficient information, the 
system displays a sequence of these three-dimensional images, 
synchronized with a real-time reading of the organ timing signal, thereby 
providing a real-time visualization of the inspected organ. At the same 

10 time, the system continues to acquire additional two-dimensional images 
and to update and elaborate the existing three-dimensional image. Hence, 
the quality of the displayed sequence of three-dimensional images, 
constantly improves. 

At this point, the physician can insert a minimal invasive surgical 

15 tool into the body of the patient. The system detects the location and 
orientation of the MPS detector mounted on the surgical tool and 
super-imposes a representation thereof, on the currently displayed 
three-dimensional image. 

The system detects movements of the patient using the MPS 

20 detector, which is attached to the patient. These movements shift the 
coordinate system of the detected organ relative to the coordinate system 
in which the two-dimensional images were acquired and the 
three-dimensional images are reconstructed. The system utilizes the 
patient MPS reading both for placing all the acquired two-dimensional 

25 images in a moving coordinate system defined by the location of the 
inspected organ, and for placingjhe location and orientation of the surgical 
tool, in that same moving coordinate system. 

According to another aspect of the invention, by removing the 
surgical tool, the system can be used merely as a pseudo real-time 

30 imaging system, used for diagnostic purposes. 
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According to a further aspect of the invention, the system 
displays the three-dimensional image sequence, using semi transparent 
stereoscopic goggles, which have an MPS sensor attached thereto. The 
system uses the goggles MPS sensor to place the location and orientation 
5 of the goggles, within the coordinate system of the patient. Accordingly, 
the system produces a visualization of the three-dimensional image 
sequence, which is perceived by the physician, as being located at the 
same place of the organ. 

The following is an example of a system and method for image 
10 acquisition, playback and minimal invasive surgery, where the inspected 
organ is a heart. 

Reference is now made to Figure 1, which is a schematic 
illustration of a multi functional three-dimensional imaging system, 
generally referenced 100, constructed and operative in accordance with a 

15 preferred embodiment of the present invention. In the example set forth in 
Figure 1, system 100 is adapted for producing a three-dimensional image 
sequence of the heart and playing it in real time synchronicity, with the 
motion of the heart. 

Three-dimensional imaging system 100 includes, a main 

20 computer 102, a two-dimensional image acquisition device 104, an ECG 
monitor 106, a medical positioning system (MPS) 108, a frame grabber 
110, a digital three-dimensional image reconstructor (D3DR) 112, an 
adaptive volumetric database (AVDB) 114, a superimposing processor 
116, a surgical tool 120, a plurality of MPS sensors 162!, 162 2 and 162 N , 

25 and a display 130. 

Two-dimensional image acquisition device 104 provides a 
two-dimensional image of an area within the body of the patient. 
Two-dimensional image acquisition device 104 can be of any type known 
in the art, such as ultra-sound, inner-vascular ultra-sound, X-ray, 

30 computerized tomography, nuclear magnetic resonance, 
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positron-emission tomography, single-photon-emission tomography, and 
the like. 

Two-dimensional image acquisition device 104 includes an 
image transducer 118. ECG-monitor continuously detects an electrical 
timing signal of the heart during inspection or surgery procedure, by 
employing a plurality of ECG-eiectrodes 128. 

Main computer 102 is connected to ECG monitor 106, MPS 
system 108, frame grabber 110, D3DR 112, superimposing processor 
116, AVDB 114 and to display 130. Two-dimensional image acquisition 
device 104 is connected to frame grabber 110. MPS system 108 includes 
an MPS transmitter (not shown) and MPS sensors 162^, 162 2 and 162 N . 

Reference is further made to Figure 2C, which is a schematic 
illustration in detail of MPS system 108, constructed and operative in 
accordance with another preferred embodiment of the present invention. 
MPS system 108 includes a location and orientation processor 150, a 
transmitter interface 152, a plurality of look-up table units 154 1; 154 2 and 
154 3 , a plurality of digital to analog converters (DAC) 156 1t 156 2 and 156 3 , 
an amplifier 158, a transmitter 160, a plurality of MPS sensors 162^ 162 2 , 
162 3 and 162 N , a plurality of analog to digital converters (ADC) 164-t, 164 2 , 
164 3 and 164 N and a sensor interface 166. 

Transmitter interface 152 is connected to location and 
orientation processor 150 and to look-up table units 154^ 154 2 and 154 3 . 
DAC units 156 l5 156 2 and 156 3 are connected to a respective one of 
look-up table units 154-,, 154 2 and 154 3 and to amplifier 158. Amplifier 158 
is further connected to transmitter 160. Transmitter 160 is also marked TX. 
MPS sensors 162 1s 162 2 , 162 3 and 162 N are further marked RX^ RX 2 , RX 3 
and RX N , respectively. 

Analog to digital converters (ADC) 164 13 164 2 , 164 3 and 164 N are 
respectively connected to sensors 162 17 162 2 , 162 3 and 162 N and to sensor 
interface 166. Sensor interface 166 is further connected to location and 
orientation processor 150. 
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Each of look-up table units 154i, 154 2 and 154 3 produces a 
cyclic sequence of numbers and provides it to the respective DAC unit 
156 1; 156 2 and 156 3 , which in turn translates it to a respective analog 
signal. Each of the analog signals is respective of a different spatial axis. 
In the present example, look-up table 154i and DAC unit 156! produce a 
signal for the X axis, look-up table 154 2 and DAC unit 156 2 produce a 
signal for the Y axis and look-up table 154 3 and DAC unit 156 3 produce a 
signal for the Z axis. 

DAC units 156-,, 156 2 and 156 3 provide their respective analog 
signals to amplifier 158, which amplifies and provides the amplified signals 
to transmitter 160. Transmitter 160 provides a multiple axis 
electromagnetic field, which can be detected by MPS sensors 162-j, 162 2 , 
162 3 and 162 N . Each of MPS sensors 162^ 162 2 , 162 3 and 162 N detects 
an electromagnetic field, produces a respective electrical analog signal 
and provides it to the respective ADC unit 164^ 164 2 , 164 3 and 164 N 
connected thereto. Each of the ADC units 164^ 164 2 , 164 3 and 164 N 
digitizes the analog signal fed thereto, converts it to a sequence of 
numbers and provides it to sensor interface 166, which in turn provides it 
to location and orientation processor 150. 

Location and orientation processor 150 analyzes the received 
sequences of numbers, thereby determining the location and orientation of 
each of the MPS sensors 162 13 162 2 , 162 3 and 162 N . Location and 
orientation processor 150 further determines distortion events and updates 
look-up tables 154-1, 154 2 and 154 3j accordingly. 

Referring back to Figure 1, image transducer 118 detects a 
plurality of two-dimensional images, each representing a slice of the 
inspected organ (i.e., the heart). Each of these two-dimensional images 
has a different spatial location and orientation. 

Frame grabber 110 grabs each detected two-dimensional image 
and provides it to main computer 102. MPS system 108 receives and 
processes data related to the location and orientation of surgical tool 1 20 
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via MPS sensor 162-1 and processes data related to the location and 
orientation of image transducer 118 via MPS sensor 162 2 . 

MPS system 108 further receives and processes data related to 
the location and orientation of the body of a patient, via MPS sensor 162 N . 
It is noted that MPS sensor 162 N is used as reference in case the patient 
moves. MPS sensor 162 N is generally attached to an inspected area of a 
patient body (reference 140). It is noted that MPS system 108 can include 
additional MPS sensors, to be used as further references, thereby 
enhancing the performance of system 100. it is noted however that other 
methods for assigning a reference point can be used such as initial 
referencing between all of the MPS sensors and strapping the patient 
during the entire procedure, analyzing the acquired images and identifying 
a recurring visual point or section therein for each of the MPS sensors 
other than the one for the transducer, and the like. 

MPS system 108 produces predetermined electromagnetic fields 
using the MPS transmitter. Each of the MPS sensors 162^ 162 2 and 162 N 
includes eiectromagnetic field detection elements, such as coils, for 
detecting the electromagnetic fields produced by MPS system 108. 

MPS system 108 processes the detected electromagnetic fields 
and provides an indication of the three-dimensional location and 
orientation of MPS sensors 162^ 162 2 and 162 N . Hence, MPS system 108 
is operative to determine the location and orientation of image transducer 
118, surgical tool 120 and a selected point on the body of the patient. 

The location and orientation of each of the captured 
two-dimensional images are directly derived from the location and 
orientation of image transducer 118. Hence, by determining the location 
and orientation of MPS sensor 162 2 , MPS system 108 can determine the 
location and orientation of each of the two-dimensional images captured 
by image transducer 118. 

ECG monitor 106 obtains and represents an electrical timing 
signal (ECG - electrocardiogram) of the inspected heart. It is noted that 
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ECG is a heart timing signal, which includes ECG cycles and represents 
the propagation of electrical currents through specific regions of the heart. 
The duration of an ECG cycle (or cardiac cycle) is defined as the time 
between two subsequent heart contractions. ECG is detected using at 
5 least two ECG-electrodes, which are placed on selected areas of the body 
of the patient (e.g., the arms, legs, chest, abdomen, and the like). 

ECG-electrodes 128 continuously obtain an electrical signal from 
the heart and provide this signal to ECG monitor 106. ECG monitor 106 
amplifies the received electrical signal, produces a graphic line tracing the 
10 electrical activity of the heart, as a function of the time, and provides this 
data in digital format to main computer 102. 

Main computer 102 receives each of the two-dimensional 
images, the respective three-dimensional location and orientation of that 
specific two-dimensional image and the organ timing signal of the heart at 
15 the time the image was captured. Main computer 102 can further receive 
the three-dimensional location and orientation of surgical tool 120. 

Main computer 102 associates each detected two-dimensional 
image, with the location and orientation information and the heart-timing 
signal. 

20 When the surgical tool 120 is located within the inspected organ, 

a two-dimensional image can include a sliced representation of a portion 
thereof. Main computer 102 receives the location and orientation of MPS 
sensor 162^ which is attached to the surgical tool and can extrapolate the 
location and orientation of a larger portion of the surgical tool, in case that 

25 portion of the surgical tool is substantially rigid. Hence, main computer 102 
can determine if that portion of surgical tool 120 is located within an area 
of the acquired two-dimensional image. Main computer 102 can discard 
this area, while updating the three-dimensional image, which the 
two-dimensional image belongs to. 

30 D3DR 112 reconstructs a three-dimensional image from 

captured two-dimensional images, having the same activity-state (e.g., for 
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each determined point of the heart timing cycle) and from the 
three-dimensional location and orientation data associated with each of 
the images. 

AVDB 114 contains the reconstructed three-dimensional images 
of the inspected organ, along with the activity-state associated therewith 
and with the location and orientation of the coordinate system thereof. The 
detected ECG sequence is further used for synchronously playing back 
the three-dimensional images, where every three-dimensional image is 
displayed when the activity-state associated therewith is substantially 
equal to real-time detected activity-state of the inspected organ. 

In case surgical tool 120 is inserted in the heart, superimposing 
processor 116 can add the three-dimensional location and orientation of 
surgical tool 120 to the reconstructed three-dimensional image. 
Alternatively, main computer 102 can extrapolate the shape of surgical tool 
120 in the coordinate system of the reconstructed three-dimensional 
image. 

Display 130 presents a three-dimensional motion picture of the 
inspected organ in synchrony therewith, which can be considered a 
pseudo real-time simulation thereof. It is noted that main computer 102 
can determine the display reference coordinate system to be any of the 
following: 

• The coordinate system of the patient, where the body of the 
patient is still and the inspected organ and the surgical tool, 
move. 

• The coordinate system of the inspected organ, where the 
inspected organ is still, and surgical tool and the rest of body 
of the patient, move. It is noted that this viewing coordinate 
system can be extremely useful in cases where the inspected 
organ exhibits rapid movement. 
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• The coordinate system of the surgical tool, where the surgical 
tool is still, and the inspected organ as well as the rest of the 
body of the patient, move. 
Reference is now made to Figures 2A and 2B. Figure 2A is an 
illustration in perspective of an inner-body radial ultrasound imaging 
system, generally referenced 170, constructed and operative in 
accordance with a further preferred embodiment of the present invention. 
Figure 2B is an illustration in perspective of a plurality of radial 
two-dimensional images of the walls of an inspected vessel, generally 
referenced 190. 

System 170 includes an Inner-body radial image transducer 172, 
a surgical tool (i.e., typically a minimal invasive surgical device) 174, MPS 
sensors 176 and 178, a mounting catheter 180 and a dilation catheter 182. 
It is noted that inner-body radial ultrasound imaging system 170 can be 
replaced with alternative ultrasound systems such as an inner-vascular 
ultrasound system (IVUS) which is discussed in further detail in Figure 12 
herein below, or other types of two-dimensional imaging systems. 

Radial image transducer 172 is mounted on mounting catheter 
180, which is further inserted in dilation catheter 182. MPS sensor 176 is 
located at a tip of mounting catheter 180 adjacent to radial image 
transducer 172. Mounting catheter 180 is inserted in dilation catheter 182. 
MPS sensor 178 is located in close proximity to the tip of surgical tool 174. 
Surgical tool 174 is further inserted in dilation catheter 182. 

Radial image transducer 172 detects a plurality of two-dimension 
images of different areas of the inspected organ (such as two-dimensional 
images 190A, 190B, 190C, 190D, 190E and 190F (Figure 2B). MPS 
system 108 (Figure 1) detects the location and orientation of radial image 
transducer 172, using sensor 176. MPS system 108 (Figure 1) further 
detects the location and orientation of surgical tool 174, using sensor 178. 
The location and orientation of two-dimensional images 190A, 190B, 
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190C, 190D, 190E and 190F (Figure 2B) are directly derived from the 
location and orientation of the transducer 172. 

As can be seen fn Figure 2B, each of the detected 
two-dimensional images 190A, 190B, 190C, 190D, 190E and 190F is 
two-dimensional representation of a different peripheral portion of the 
inspected area within the inspected organ and its vicinity. Radial image 
transducer 172 provides the detected two-dimension images 190A, 190B, 
190C, 190D, 190E and 190F to two-dimensional image acquisition device 
104 (Figure 1). The System 100 associated between each 
two-dimensional image and the respective location and orientation. 

Reference is now made to Figure 3, which is a schematic 
illustration of a two-dimensional image, generally referenced 190, in a 
given coordinate system, generally referenced 186. Figure 3 is mainly 
used for visualizing the terms "location" and "orientation" of the 
two-dimensional image 190 in coordinate system 186. 

The location and orientation of each two-dimensional image 190 
are determined in the coordinate system 186 (X, Y and Z). System 100 
determines a selected point in each captured two-dimensional image, 
which is to be the reference point for that image. In the example set forth 
in Figure 3, the center of the image is determined to be the reference 
location point thereof. A unit vector extending from that point, 
perpendicular to the plane of that image determines the orientation of that 
image. 

Each detected two-dimensional image 190 is taken in a specific 
location (X', Y* and Z') and a specific orientation (angles a, (3 and %). 
Vector 192 extends from a selected point 194 of the image 190. The 
coordinates of this point X', Y' and Z' determine the specific 
three-dimensional location of the image 190 in the coordinate system 186. 
Angles a, p and % are the angles between the vector 192 and each of the 
axes X, Y and Z, accordingly. Thereby, vector 192 determines the specific 
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three-dimensional orientation of the image 190 in the coordinate system 
186. 

Reference is now made to Figure 4, which is an illustration in 
perspective of a plurality of two-dimensional images, generally referenced 
190, and an organ timing signal, generally referenced 188. In the example 
set forth in Figure 4, organ timing signal is an ECG signal. 

The ECG signal can be used for synchronizing the detection 
procedure of two-dimensional images 190A, 190B, 190C, 190D, 190E, 
190F, 190G, 190H, 1901, 190J, 190K, 190L, 190M, 190N, 190O, 190P, 
190Q, 190R and 190S, where each image is taken at a predetermined 
position in the organ timing signal. Two-dimensional images 190A, 190B, 
190C, 190D, 190E, 190F, 190G, 190H, 1901, 190J, 190K, 190L, 190M, 
190N, 190O, 190P, 190Q, 190R and 190S are detected at predefined 
points in time to, t 1t t 2 , t 3 , U, U, ^, h, h, t 9 , t i0 , tn, t 12 , t 13 , t 14 , t 15 , t 16t t 17 and 
t 18 , respectively. T denotes the cycle duration of ECG signal 188 (e.g., the 
time interval between the time points toandta). Each point p 0 , Pi, p 2 , p 3 , p 4 , 
p 5 , p 6 , p 7 , Pa, Pg, P10, Pii, Pi2, Pis, Pi4, Pis, Pie. P17 and p 18 denotes a 
specific position on the ECG timing signal and is associated with specific 
activity-state of the heart. 

In this example, two-dimensional images are detected 
continuously at a rate of eight images per ECG cycle into predetermined 
points in each heart cycle. Each point p 0 , Pi, p 2 , p 3 , p4, Ps, Ps and p 7 
denotes a specific position on the first ECG cycle, each point p 8 , pg, P10, 
Pur P12- Pi3, Pi4 and pi 5 denotes a specific position on the second ECG 
cycle, and the like. Points p 8 and p 16 have the same specific position on 
the ECG timing signal, as point p 0l and hence are associated with the 
same activity-state. Points p 9 and p 17 have the same specific position on 
the ECG timing signal, as point p^, and hence are associated with the 
same activity-state. Points p 10 and p 18 have the same specific position on 
the ECG timing signal, as point p 2 , and hence are associated with the 
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same activity-state. Accordingly, each detected two-dimensional image is 
associated with a specific activity-state of the heart. 

Reference is now made to Figures 5A, 5B, 5C and 5D. Figure 
5A is a schematic illustration of a plurality of three-dimensional volumes, 
generally referenced 200, according to another preferred embodiment of 
the present invention. Figure 5B is a schematic illustration of some of the 
three-dimensional volumes of Figure 5A, at a later stage of image 
reconstruction. Figure 5C is a schematic illustration of a selected 
three-dimensional volume of Figure 5A, going through a procedure of 
image updating. Figure 5D is a schematic illustration of a two-dimensional 
image, which includes foreign object information. 

With reference to Figure 5A, each of the three-dimensional 
volumes 200 is associated with a selected one of the specific positions in 
the organ timing signal cycle, and hence is associated with the respective 
activity-state. In the present example, three-dimensional volumes 200A, 
200B, 200C and 200D are associated with organ timing signal cycle 
locations T, %T, 1 /4T and 3 AT, respectively. 

Each of the three-dimensional volumes 200A, 200B, 200C and 
200D is used for reconstructing a three-dimensional image for a selected 
location in the organ timing signal cycle, and hence for the respective 
activity-state. Main computer 102 (Figure 1) sorts the two-dimensional 
images according to the timing position of the image on the ECG signal 
(i.e., specific activity-state). 

In the present example, volume 200A includes two-dimensional 
images 190A, 1901 and 190Q (Figure 4), which were detected at time 
points t 0 , t 8 and t 16 , respectively. The position in the organ timing signal 
cycle of these images is T. Volume 200B includes two-dimensional images 
190C, 190K and 190S (Figure 4), which were detected at time points t 2) t 10 
and t 18 , respectively. The position in the organ timing signal cycle of these 
images is % T. Volume 200C includes two-dimensional images 190E and 
190M (Figure 4), which were detected at time points and t 12 , 
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respectively. The position in the organ timing signal cycle of these images 
is !4 T. Volume 200D includes two-dimensional images 190G and 190O 
(Figure 4), which were detected at time points t 5 and t 14 , respectively. The 
position in the organ timing signal cycle of these images is Z A T. 

At this point, volume 200A contains information relating to the 
two-dimensional images that were stored therein, while portions of volume 
200A remain at zero value, since no two-dimensional image is related 
thereto. D3DR 112 analyzes the content of three-dimensional volume 
200A and attempts to determine the value of some of these zero value 
portions, for example, by means of extrapolation. With reference to Figure 
5B, D3DR 112 (Figure 1) reconstructs image 330A within 
three-dimensional volume 200A. Similarly, D3DR 112 reconstructs image 
330C within three-dimensional volume 200C. 

System 100 (Figure 1) updates the three-dimensional image 
330A in real time. Main computer 102 (Figure 1) continuously receives 
two-dimensional images, associated with a location and orientation thereof 
and an organ activity-state. Main computer 102 (Figure 1) provides each of 
these two-dimensional images to D3DR 112 (Figure 1) together with the 
three-dimensional volume, associated with the same organ activity-state. 
D3DR 112 updates the three-dimensional volume according to the values 
of the new two-dimensiona! images. 

The update procedure can be performed in many ways. 
According to one aspect of the invention, a new value in a selected 
three-dimensional pixel (voxel) replaces an old value. According to another 
aspect of the invention, an updated voxel value includes a combination 
(linear or otherwise) of the old voxel value (i.e., which already exists in the 
three- dimensional volume) and the newly acquired value (i.e., received 
from the two-dimensional image). It is noted that system 1 00 can operate 
either using polygonal or voxel representations- 
According to a further aspect of the invention, each of the voxels 

in the three-dimensional volume includes various attributes such as if the 
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current value thereof, was provided from an acquired image, or was 
calculated in the process of reconstructing the three-dimensional image, 
by means of extrapolation. In this case, a newly acquired value is 
preferred over a calculated one. 

With reference to Figure 5C, D3DR 112 receives a new 
two-dimensional image 190Y, which is associated with an organ activity 
state of t=T. D3DR 112 updates the respective three-dimensional volume 
200A and the image therein 330A, thereby producing an updated image 
330A UPDATED . 

In case where a surgical tool 120 (Figure 1) is inserted in the 
inspected organ, the system 100 excludes a fragment of the 
two-dimensional image, which contains a representation of the surgical 
tool 120. Main computer 102 (Figure 1) modifies the two-dimensional 
image by excluding these fragments (e.g. by introducing null values to 
those fragments). D3DR 112 analyzes the modified two-dimensional 
image and does not update the respective portions in the respective 
three-dimensional volume. 

System 100 incorporates the ability to update the 

three-dimensional image sequence of the inspected object, in real time, 

even in the presence of foreign objects such as surgical tool 174 (Figure 

2A). According to a preferred embodiment of the invention, main computer 

102 can determines according to the location and orientation of an 

acquired image and of surgical tool 174, if the surgical tool was included in 

the slice represented by the acquired image. With reference to Figure 5D, 

two-dimensional image 190Z incorporates a small section 198, which is a 

representation of a portion of surgical tool 120, as determined from the 

location and orientation of MPS sensor 162^ Two-dimensional image 

190Z and section 198 define a section 199, in which section 198 is 

excluded from two-dimensional image 190Z. According to real-time 

updating procedure of the three-dimensional image sequence which was 

described in connection with Figure 5D, system 100 updates only section 
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199 to the respective three-dimensional image. Hence, when the surgical 
tool is removed from the body of the patient, it leaves no traces in the 
reconstructed three-dimensional images. 

ft is noted that each ECG cycle consists of a period of relaxation, 
named a diastole followed by a period of contraction named a systole. 
Duration of the ECG cycle is defined as a time between two subsequent 
heart contractions. According to a further preferred embodiment, the ECG 
cycle is evenly divided by N, where N denotes the number of 
three-dimensional images in the final image sequence. 

Reference is now made to Figure 6, which is a schematic 
illustration of a method for operating the three-dimensional imaging 
system 100, operative in accordance with a further preferred embodiment 
of the presented invention. 

In step 230, the timing signal of an inspected organ is detected. 
The detection is performed by a medical monitoring device, which is 
selected according to the inspected organ. For example, if the inspected 
organs are blood vessels, then the medical device can be any 
conventional stent delivering system, a balloon mechanism, and the like. 

If the inspected organ is the heart, then the medical device is an 
ECG monitor. If the inspected organs are the lungs, then the medical 
device is a respiratory rate monitor. Special devices can be constructed for 
detecting the movement of the eye lid, the eye, and the like. For example, 
an MPS sensor can be attached to the eye fid for detecting the movement 
thereof. With reference to Figure 1, ECG monitor 106 detects the organ 
timing signal through ECG-electrodes 128. 

In step 232, a plurality of two-dimensional images of the 
inspected organ is detected. With reference to Figure 1, two-dimensional 
image acquisition device 104 detects a plurality of two-dimensional images 
of the inspected organ through image transducer 118. 

In step 234, the three-dimensional location and orientation of the 

image detector is detected. With reference to Figure 1, MPS system 108 
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detects the three-dimensional location and orientation of the image 
detector using MPS sensor 162 2 , mounted thereon. 

In step 236, each detected two-dimensional image is associated 
with the location and orientation information thereof and the organ timing 
signal at the time the two-dimensional image was taken. With reference to 
Figure 1, main computer 102 receives the ECG signal, the acquired 
two-dimensional images and the location and orientation of each 
two-dimensional image. Main computer 102 associates each detected 
image with the location and orientation information thereof and the 
organ-timing signal. 

In step 238, the location and orientation of a surgical tool are 
detected. With reference to Figure 1, MPS system 108 detects the location 
and orientation of surgical tool 120, via MPS sensor 162^ 

In step 240, the two-dimensional images are modified by 
discarding a portion thereof, which represents the surgical tool. It is noted 
that two-dimensional images which are located in planes which do not 
intersect the surgical tool, do not include any image respective thereof and 
hence remain unchanged. With reference to Figures 1 and 5D, main 
computer 102 estimates the portion within an acquired two-dimensional 
image, such as two-dimensional image 190Z, that might include a 
representation of the image of surgical tool 120. Main computer 102 
performs that estimation according to the detected location and orientation 
of surgical tool 120. Main computer 102 determines a three-dimensional 
space which is occupied by surgical tool 120, according to the information 
which MPS sensor 162-1 acquires and according to data respective of the 
physical dimensions of surgical tool 120. Main computer 102 calculates an 
intersection area (e.g., portion 198 as illustrated in Figure 5D), in which an 
acquired two-dimensional image (e.g., reference 190Z in Figure 5D) and 
that three-dimensional space intersect. Main computer 102 discards that 
intersection area for example, by changing the values thereof to null 
values. Discarding of the image of surgical tool 120 from the 
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two-dimensional images, is necessary in order to reconstruct a 
three-dimensional image of the inspected organ, free of artifacts such as 
the image of surgical tool 120. 

In step 242, a three-dimensional image is reconstructed. The 
reconstruction includes three procedures. The first procedure is sorting the 
two-dimensional images into groups, according to their respective timing 
position (i.e., activity-state) within the organ timing signal. With reference 
to Figure 1 main computer 102 sorts the two-dimensional images 
according to the image timing position in the organ timing signal cycle. 

The second procedure is the placing of all of the two- 
dimensional images of a selected group in a three-dimensional virtual 
volume (such as a three-dimensional matrix), according to their respective 
location and orientation. With reference to Figure 1, main computer 102 
stores each two-dimensional image in a selected one of the 
three-dimensional virtual volumes 200 (Figure 5A) within adaptive 
volumetric database 114. 

The third procedure is filling the missing parts in the 
three-dimensional virtual volumes, for example, by means of interpolation. 
With reference to Figure 1, D3DR 112 reconstructs a three-dimensional 
image in each of the three-dimensional virtual volumes 200 (Figure 5B), 
from the two-dimensional images stored therein. 

In step 244, a three-dimensional image is selected according to 
the organ timing signal detected in step 230. With reference to Figures 1 
and 4, ECG monitor 106 detects the organ timing signal 188 of the 
inspected organ via ECG-electrodes 128. Main computer 102 selects 
three-dimensional images according to a real time detected organ timing 
signal. The selected three-dimensional image has to be associated with 
the position in the organ timing signal cycle, as the position of the real time 
detected organ timing signal cycle. 

In step 246, a representation of the surgical tool is added to the 
selected three-dimensional image. With reference to Figure 1, MPS 
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sensor 162-, detects the location and orientation of surgical tool 120. 
Superimposing processor 116 adds a representation (e.g., a symbol) of 
surgical tool 120, to the selected three-dimensional image. Superimposing 
processor 116 adds the representation according to the location and 
orientation of surgical tool 120, which MPS sensor 162 1 detects in step 
238. Since the location and orientation of the surgical tool and the location 
and orientation of the acquired images, are all detected using the same 
MPS system, all reside in a single coordinate system and hence do not 
have to be correlated. 

In step 248, the selected three-dimensional image is displayed. 
With reference to Figures 1 and 8, display 130 displays the selected 
three-dimensional images in a sequence, according to the real time 
detected organ timing signal cycle. For example, the operator can view a 
video of the heart of the patient, which corresponds to the real time 
detected heartbeat of the patient and at the same time, hear this heartbeat 
through a stethoscope. 

Reference is now made to Figures 7A and 7B. Figure 7A is a 
schematic illustration of a real-time three-dimensional display playback 
system, generally referenced 270, constructed and operative in 
accordance with another preferred embodiment of the present invention. 
Figure 7B is a schematic illustration of the goggles of Figure 7A, displaying 
a three-dimensional image of the heart of a patient. 

With reference to Figure 7A, system 270 includes a 
superimposing processor 272, an adaptive volumetric database (AVDB) 
274, an ECG monitor 278, a display 284, a synchronization processor 286, 
a medical positioning system (MPS) 288, a dilation catheter 280 and a 
plurality of MPS sensors 294, 296 and 298. Display 284 includes a 
Tenderer 276 and goggles 282. A surgical tool 290 is inserted into dilation 
catheter 280. 

Superimposing processor 272 is connected to AVDB 274, 
Tenderer 276 and to MPS system 288. Renderer 276 is further connected 
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AVDB 274 and to goggles 282. MPS system 288 is further connected to 
dilation catheter 280, renderer 276 and to MPS sensors 294, 296 and 298. 
Synchronization processor 286 is connected to AVDB 274 and to ECG 
monitor 278. 

ECG monitor 278 detects the organ timing signal of the 
inspected organ and provides this signal to synchronization processor 286. 
The detected organ timing signal is used for synchronizing between a 
sequence of three-dimensional images and the movement of the 
inspected heart. 

Synchronization processor 286 analyzes the ECG signal, and 
determines activity-states therein. Synchronization processor 286 provides 
a retrieval command to AVDB 274, to retrieve an image record, according 
to the currently detected activity-state. 

AVDB 274 contains a sequence of three-dimensional images of 
the inspected organ, along with an activity-state associated therewith and 
with the location and orientation of the coordinate system thereof. It is 
noted that this sequence of three-dimensional images can be acquired 
using system 100 (Figure 1) or any other system for acquiring 
three-dimensional images (e.g., MRI, X-rays, and the like). 

AVDB 274 selects a three-dimensional image of the organ, 
which is associated with the specific activity-state according to the 
received retrieval command. This three-dimensional image can be 
rendered and displayed as is, in synchrony with the organ timing signal. In 
the example, where the inspected organ is the heart, the physician is 
presented with an image sequence, which is real-time synchronized with 
the ECG signal. The physician, as he uses his stethoscope to hear the 
heart beats, can see a moving visual representation of the heart, at the 
same time. 

Renderer 276 can render the three-dimensional image 
according to reference coordinates of surgical tool 290, reference 
coordinates of the inspected organ, or reference coordinates of the body 
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of the patient. The selected reference coordinates define a stationary 
reference coordinate system, in which all other objects may move. 

For example, if the coordinates of surgical tool 290 are selected 
as reference, then renderer 276 renders the three-dimensional image so 
that the surgical tool is stationary and the heart moves relative thereto. 
Accordingly, when the physician moves surgical tool 290 relative to the 
inspected organ, she observes a stationary representation of surgical tool 
290, while the inspected organ exhibits movement due to the motion of the 
inspected organ relative to the representation of surgical tool 290. 

On the contrary, if the coordinates of the inspected organ are 
selected as reference, then renderer 276 renders the three-dimensional 
image so that the inspected organ is stationary and surgical tool 290 
moves relative thereto. Accordingly, when the physician moves surgical 
tool 290 relative to the inspected organ, she observes a stationary image 
of the inspected organ, while surgical tool 290 exhibits movement due to 
the motion of the representation or surgical tool 290 relative to the 
inspected organ. 

MPS system 288 includes an MPS transmitter (not shown) and 
MPS sensors 294, 296 and 298. MPS system 288 determines the location 
and orientation of surgical tool 290, using sensor 294 mounted thereon. 
MPS system 288 determines the location and orientation of the point of 
view of the user, using MPS sensor 296 (Figure 7B), mounted on goggles 
282. MPS system 288 determines the location and orientation of the body 
of the patient using MPS sensor 298, attached thereto. 

Superimposing processor 272 receives the selected 
three-dimensional image of the heart from AVDB 274. Superimposing 
processor 272 further receives parameters related to the location and 
orientation of surgical tool 290 and parameters related to the location and 
orientation of reference points on the body of the patient, from MPS 
system 288. 
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Superimposing processor 272 uses the location and orientation 
of the reference points to align the coordinate system of the 
three-dimensional images with the coordinate system of surgical tool 290. 
Superimposing processor 272 adds a representation of surgical too! 290 to 
the three-dimensional image of the heart and provides this image to 
renderer 276. 

Renderer 276 receives parameters related to the location and 
orientation of goggles 282 and the parameters related to the location and 
orientation of the reference points of the body of the patient, from MPS 
system 288. Renderer 276 uses these parameters to determine a viewing 
plane, and renders the three-dimensional images to that viewing plane. 
Renderer 276 provides the rendered images to goggles 282. 

Goggles 282 (Figure 7B) are preferably see-through, so that 
they are semi-transparent, in a way that the physician can actually see the 
body of the patient as well as an image projected on the goggie screens. 
System 270 (Figure 7A) determines a projection plane on the body of the 
patient, according to the location and orientation of the body of the patient 
and of goggles 282. System 270 displays the three-dimensional image 
sequence using goggles 282 so that it is perceived to be within the body of 
the patient, as actually seen by the physician through the goggles 282. 

Reference is now made to Figure 8, which is an illustration in 
perspective of a cyclical sequence of three-dimensional images, according 
to a further preferred embodiment of the present invention. The sequence 
is presented, image by image to a user, so as to produce an illusion of a 
moving image. The intermediate presented three-dimensional image is 
selected according to a real-time detected specific location in the organ 
timing signal cycle. 

With reference to Figure 7, AVDB 274 selects three-dimensional 
images continuously, at a rate of eight images per each heart cycle. 
Synchronization processor 286 provides a retrieval command to AVDB 
274, according to the currently detected specific position in the organ 
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timing signal cycle. AVDB 274 retrieves three-dimensional images 330A, 
330B, 330C, 330D and 330H according to selected points ^=0, ti=1/8T, 
t 2 =2/8T, t 3 =3/8T, t4=4/8T and t 7 =7/8T within the heart timing signal (Figure 
4), respectively. As stated above, each three-dimensional image 330 is 
associated with the specific activity-state of the heart. Hence, playing 
images 330A, 330B, 330C, 330D and 330H in cyclic sequence provides a 
pseudo realistic illustration of the motion of the organ (i.e., due to the 
repetitive nature of the periodic cardiac motion). 

Reference is now made to Figure 9, which is a schematic 
illustration of a superimposing process, operative in accordance with 
another preferred embodiment of the present invention. The system 270 
(Figure 7) introduces a representation of the currently used surgical tool 
328 to each selected image, such as image 330A. This representation can 
either be a minimal one (e.g., in the form of a cursor) or an elaborated one 
which provides pseudo realistic visualization of that surgical tool. 

The location and orientation of three-dimensional image 330A 
are determined in a coordinate system 326 (X, Y and Z). Similarly, the 
location and orientation of a MPS sensor 318, and hence the location and 
orientation of a surgical tool 328 are also determined in the coordinate 
system 326 (X, Y and Z). Accordingly, a real-time representation of 
surgical tool 328 can be added to tree-dimensional image 330A. With 
reference to Figure 7, superimposing processor 272 adds the 
representation of surgical tool 328 to selected three-dimensional image 
330A, which was reconstructed in thee-dimensional volume 200A. It is 
noted that the surgical tool can be any conventional tool (e.g., clamp, laser 
cutter, brush, catheter, stent, balloon, pace maker electrode, solution 
dispensing unit, neuron electrode, substance collection unit, surgical 
delivery tool (e.g., for delivering genes, drugs, devices and the like), and 
the like), for example, a device delivery tool can be a medical tool for 
delivery of a medical device, such as a permanent stent, a removable 
stent, and the like, to the body of the patient. 
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The present invention allows the physician to operate on a 
dynamically moving object such as the heart, with a real-time 
representation surgical tool that he is using. 

Reference is now made to Figure 10, which is a schematic 
illustration of a method for operating system 270, operative in accordance 
with a further preferred embodiment of the present invention. In step 300, 
an organ timing signal of the inspected organ is detected. According to the 
present invention, the system 100 (Figure 1) includes a medical monitoring 
device, which is selected according to the inspected organ. Such device 
detects a time dependent signal of the organ, which is associated with an 
organ movement. With reference to Figure 7, ECG monitor 278 detects 
the heart-timing signal. 

In step 302, a three-dimensional image of the organ (e.g., the 
heart), associated with a selected timing point within the organ timing 
signal is selected. With reference to Figure 7, AVDB 274 selects the 
three-dimensional image of the organ. 

In step 304, the location and orientation of a surgical tool, are 
detected. Detection of the location and orientation of a surgical tool can be 
performed by methods known in the art, such as magnetic fields, 
ultrasound triangulation or radiation, inertial sensor — dead reckoning 
sensor, and the like. With reference to Figure 7, MPS system 288 detects 
the location and orientation of surgical tool 290, using sensor 294 mounted 
thereon. 

In step 306, a representation of the location and orientation of 
the surgical tool is superimposed onto the selected three-dimensional 
image. With reference to Figure 7, superimposing processor 272 
superimposes parameters related to location and orientation of the 
surgical tool onto the selected three-dimensional image. 

In step 308, the location and orientation of the point of view of 
the user, are detected. The location and orientation of the point of view of 
the user are derived from the location and orientation of goggles 282. 
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Parameters of the location and orientation of goggles 282 determine a 
viewing plane of the user. System 270 can determine two adjacent viewing 
planes, one for each LCD element of the goggles (one for each eye). With 
reference to Figure 7, MPS system 288 detects the location and 
orientation of goggles 282, using sensor 296 mounted thereon. It is noted 
that more than one display unit (i.e., goggles) can be introduced to the 
system, employing a different MPS sensor, mounted thereon. 

In step 310, the selected three-dimensional image is rendered 
according to the location and orientation of the point of view of the user. 
With reference to Figure 7, Tenderer 276 renders the selected 
three-dimensional image according to the location and orientation of the 
point of view. 

In step 312, a rendered three-dimensional image sequence (i.e., 
a three-dimensional motion picture) is displayed. The three-dimensional 
motion picture of the inspected organ can be displayed on any type of 
display, monoscopic, stereoscopic or holographic (e.g., a video monitor, 
goggles, holographic generator, and the like). With reference to Figure 7, 
goggles 282 displays a rendered three-dimensional image sequence. 

Reference is now made to Figure 11, which is an illustration in 
perspective of an inner-vascular imaging and surgery system, generally 
referenced 350, constructed and operative in accordance with another 
preferred embodiment of the invention. 

Inner-vascular imaging and surgery system 350 includes an 
inner-vascular ultrasound system (IVUS) transducer 352, a surgical tool 
(i.e., typically a minimal invasive surgical device) 354, MPS sensors 356 
and 358, a mounting catheter 360 and a dilation catheter 362. IVUS 
transducer 352 is mounted on mounting catheter 360. It is noted that other 
IVUS devices, such as ones which include rotary acoustic mirrors, are also 
applicable to this embodiment. 

IVUS transducer 352 produces ultrasound waves and directs 
them at the wall of a tubular organ (not shown), covering a surface 
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referenced 364. Surface 364 reflects a portion of the ultrasonic waves 
directed thereto. IVUS transducer 352 detects these reflected ultrasonic 
waves and provides a respective signal to an image processing system 
(not shown) connected thereto. A conventional image processing system 
reconstructs a single three-dimensional image from all of the 
two-dimensional image, according to the location and orientation of each 
of them. According to the present invention, image processing system 
reconstructs a plurality of three-dimensional images, each for a different 
position in the timing cycle of the organ timing signal. 

It is noted, that the IVUS is used for imaging the interior of 
coronary arteries during diagnosis, clinical treatment and performing 
research. This system provides a comprehensive and detailed 
understanding of the coronary arteries (or other blood vessels). A 
conventional IVUS detector is, for example, the Ultra-Cross system, 
manufactured by Boston Scientific Scimed or the In-Vision system, 
manufactured by Jomed USA (aka Endosonics) both companies located in 
San-Diego, California USA. These systems are mainly used for slicing 
images of coronary images. 

The present invention provides a novel structure, for producing a 
qualitative model of the blood vessel, using a combination of conventional 
IVUS elements and the unique miniature MPS sensor. According to a 
further preferred embodiment of the invention, the IVUS image sequence 
can be visually stabilized using the ECG signal. A selected time point in 
the ECG signal cycle, determines the momentary local blood pressure in 
the blood vessel, which in turn determines the momentary diameter of that 
blood vessel for that position within the cycle. Hence, the visual diameter 
of the blood vessel can be adjusted artificially, by means of image 
processing, so as to present a steady image of the blood vessel during the 
surgical procedure. 

In general, the IVUS system produces a "road map" in which the 
surgeon navigates. Other imaging methods can also be used for 
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producing such a road map, which can be stored in the volumetric 
database and played back according to the method of the invention, such 
as 3D ANGIUM, and the like. 

Reference is now made to Figure 12, which is a schematic 
illustration of a multi function two-dimensional imaging system, generally 
referenced 400, constructed and operative in accordance with a further 
preferred embodiment of the present invention. In the example set forth in 
Figure 12, system 400 is adapted for producing a two-dimensional image 
sequence of the heart and playing it in real time in synchrony with the 
motion of the heart. 

Two-dimensional imaging system 400 includes, a main computer 
402, a two-dimensional image acquisition device 404, an ECG monitor 
406, a medical positioning system (MPS) 408, a frame grabber 410, an 
image database 414, a superimposing processor 416, a surgical tool 420, 
a plurality of MPS sensors 432!, 432 2 and 432 N , and a display 430. 

Two-dimensional image acquisition device 404 includes an 
image transducer 418. ECG monitor 406 continuously detects an electrical 
timing signal of the heart during inspection or surgery, by employing a 
plurality of ECG electrodes 428. 

Main computer 402 is connected to ECG monitor 406, MPS 
system 408, frame grabber 410, superimposing processor 416 and to 
display 430. Two-dimensional image acquisition device 404 is connected 
to frame grabber 410. MPS system 408 includes an MPS transmitter (not 
shown) and sensors MPS 432^, 432 2 and 432 N . 

System 400 is directed at capturing and playing a 
two-dimensional image sequence of the inspected organ, with a 
superimposed representation of the projection of the surgical tool 420. 
Transducer 418 detects a plurality of two-dimensional images of the 
inspected organ and provides them to two-dimensional image acquisition 
device 404, which further transfers them to frame grabber 410. Frame 
grabber 410 grabs each detected two-dimensional image and provides it 
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to main computer 402, which stores them in image database 414, along 
with an organ timing signal, as received from the ECG monitor 406. The 
images can be used at any time to produce a two-dimensional cyclic 
image sequence of a selected plane of the inspected organ. System 400 
can synchronize this image sequence with a real-time reading of the timing 
signal of the inspected organ, using procedures similar to those described 
above. 

The example set forth in Figure 12 includes MPS sensor 432 2 , 
which detects the location and orientation of transducer 41 8 for each of 
the acquired images and MPS system 408, which determines if all of the 
acquired images reside on the same plane. If not, then MPS system 408 
can indicate to the user a detected deviation from a given plane, either 
visually, audibly or mechanically (e.g., by means of vibration, and the like). 
It is noted that a simpler version of system 400, according to a further 
preferred embodiment, does not include an MPS sensor attached to the 
transducer. 

MPS sensor 432i detects the location and orientation of surgical 
tool 420. MPS system 408 determines the location and orientation of 
surgical tool 420 and provides this information to main computer 402, 
which in turn provides it to super-imposing processor 416. Superimposing 
processor 416 determines a representation in space, of the surgical tool 
420, derives a projection thereof, onto the plane of the detected images 
and superimposes that projection with each of the images in real time. 
Display 430 displays superimposed image sequence for the user. 

Superimposing processor 416 can add additional information to 
the super-imposed sequence, such as the location of the surgical tool, 
above or under the viewed plane. For example, portions of the surgical 
tool which are located on one side of the viewed plane shall be indicated 
using red color, portions of the surgical tool which are located on the other 
side of the viewed plane shall be indicated using blue color, and portions 
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of the surgical tool which are located on viewed plane shall be indicated 
using red color. 

It is noted that although two-dimensional system 400 provides 
less visual information than system 100 of Figure 1, it requires significantly 
less processing power and storage area and hence can be manufactured 
at considerably lower costs. 

!t will be appreciated by persons skilled in the art that the 
present invention is not limited to what has been particularly shown and 
described hereinabove. Rather the scope of the present invention is 
defined only by the claims, which follow. 
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